Oracle Clusterware and Agents
-----------------------------
	Oracle Clusterware is software that enables multiple servers to operate together 
as if they are a single server. Oracle Clusterware is used to provide high 
availability and scalability to Oracle Databases as well as custom applications. 
Anything that is managed by Oracle Clusterware is termed as a "resource". A 
resource could be a database, a service,a listener, an application process and so 
on. Oracle Clusterware's policy engine manages the resources through a process called
the agent. The agent manages the application/resource through the application specific 
user code. Oracle Clusterware contains a special shared library (libagfw) which allows 
users to plug-in application specific actions using a well defined interface.
The following sections describe how to build an agent using Oracle Clusterware's
agent framework interface.

Action Entry Points
-------------------
      Action entry points refer to user defined code that needs to be executed whenever
an action has to be taken on a resource (start resource, stop resource etc.). For every
resource type, Clusterware requires that action entry points are defined for the following 
actions:

	start : Actions to be taken to start the resource
	stop  : Actions to gracefully stop the resource	
	check : Actions taken to check the status of the resource
	clean : Actions to forcefully stop the resource.

These action entry points can be defined using C++ code or in script. If any of these 
actions are not explicitly defined, Clusterware assumes by default that they are defined in a 
script. This script is located via the ACTION_SCRIPT attribute for the resource type. Hence it is 
possible to have hybrid agents, which define some action entry points using script and other
action entry points using C++. It is possible to define action entry points for other actions 
too (e.g. for changes in attribute value) but these are not mandatory.
    
Sample Agents
-------------
	Consider a file as the resource that needs to be managed by Clusterware. An 
agent that manages this resource has the following tasks:
      On startup         : Create the file.
      On shutdown        : Gracefully delete the file.
      On check command   : Detect whether the file is present or not.
      On clean command   : Forcefully delete the file.

  To describe this particular resource to Oracle Clusterware, a specialized resource type is 
first created, that contains all the characteristic attributes for this resource class. In this
case, the only special attribute to be described is the filename to be monitored. This can be done 
with the crsctl command. While defining the resource type, we can also specify the ACTION_SCRIPT
and AGENT_FILENAME attributes. These are used to refer to the shell script and executables,
that contain the action entry points for the agents.

Once the resource type is defined, there are several options to write a specialized agent which 
does the required tasks - the agent could be written as a script, as a C/C++ program or as a hybrid. 
Examples for each of them are given below.

Option 1: Shell script agent
----------------------------
The file demoActionScript is a shell script which already contains all the required action
entry points and can act as an agent for the file resource. To test this script, the following 
steps need to be performed:
(1) Start the Clusterware installation.
(2) Add a new resource type using the crsctl utility as below

       crsctl add type test_type1 -basetype cluster_resource -attr \
    "ATTRIBUTE=PATH_NAME,TYPE=string,DEFAULT_VALUE=default.txt,    \
     ATTRIBUTE=ACTION_SCRIPT,TYPE=string,DEFAULT_VALUE=/path/to/demoActionScript" 

    Modify the path to the file appropriately. This adds a new resource 
    type to Clusterware. Alternately, the attributes can be added in a text file
    which is passed as a parameter to the CRSCTL utility. Refer to the documentation 
    for details on how this is done.

(3) Add new resources to the cluster using the crsctl utility. The commands to do 
    this are: 

    crsctl add resource r1 -type test_type1 -attr "PATH_NAME=/tmp/r1.txt"
    crsctl add resource r2 -type test_type1 -attr "PATH_NAME=/tmp/r2.txt"

    Modify the PATH_NAME attribute for the resources as needed. This adds resources named r1 
    and r2 to be monitored by clusterware. Here we are overriding the default value for the 
    PATH_NAME attribute for our resources. 
(4) Start/stop the resources using the crsctl utility. The commands to do this are:

    crsctl start res r1
    crsctl start res r2
    crsctl check res r1
    crsctl stop res r2
    etc...

    The files /tmp/r1.txt and /tmp/r2.txt get created and deleted as the resources 
    r1 and r2 get started and stopped.

Option 2: C++ agent
-------------------
demoagent1.cpp is a sample C++ program that has similar functionality to the shell 
script above. This program also monitors a specified file on the local machine. To 
test this program, the following steps need to be performed:

(1) Compile the C++ agent using the provided source file demoagent1.cpp and makefile. 
    The makefile needs to be modified based on the local compiler/linker paths and 
    install locations. The output will be an executable named demoagent1
(2) Start the Clusterware installation.
(3) Add a new resource type using the crsctl utility as below

      crsctl add type test_type1 -basetype cluster_resource -attr \
    "ATTRIBUTE=PATH_NAME,TYPE=string,DEFAULT_VALUE=default.txt,   \
     ATTRIBUTE=AGENT_FILENAME,TYPE=string,DEFAULT_VALUE=/path/to/demoagent1"

    Modify the path to the file appropriately. This adds a new resource 
    type to Clusterware.

(4) Create a new resource based on the type that is defined above. The commands are as 
    follows:
    
    crsctl add res r3 -type test_type1 -attr "PATH_NAME=/tmp/r3.txt"
    crsctl add res r4 -type test_type1 -attr "PATH_NAME=/tmp/r4.txt"

    This adds resources named r3 and r4 to be monitored by Clusterware.
(5) Start/stop the resource using the CRSCTL utility. The commands to do so are:

    crsctl start res r3
    crsctl start res r4
    crsctl check res r3
    crsctl stop res r4
    etc...

    The files /tmp/r3.txt and /tmp/r4.txt get created and deleted as the resources 
    get started and stopped.

Option 3: Hybrid agent
----------------------
demoagent2.cpp is a sample C++ program that has similar functionality to the shell 
script above. This program also monitors a specified file on the local machine. However,
this program defines only the CHECK action entry point - all other action entry points 
are left undefined and are read from the ACTION_SCRIPT attribute. To test this program, 
the following steps need to be performed:

(1) Compile the C++ agent using the provided source file demoagent2.cpp and makefile. 
    The makefile needs to be modified based on the local compiler/linker paths and 
    install locations. The output will be an executable named demoagent1
(2) Start the Clusterware installation.
(3) Add a new resource type using the crsctl utility as below

      crsctl add type test_type1 -basetype cluster_resource -attr \ 
    "ATTRIBUTE=PATH_NAME,TYPE=string,DEFAULT_VALUE=default.txt,   \
     ATTRIBUTE=AGENT_FILENAME,TYPE=string,DEFAULT_VALUE=/path/to/demoagent2, \ 
     ATTRIBUTE=ACTION_SCRIPT,TYPE=string,DEFAULT_VALUE=/path/to/demoActionScript"

    Modify the path to the files appropriately. This adds a new resource 
    type to Clusterware.

(4) Create new resources based on the type that is defined above. The commands are as 
    follows:
    
    crsctl add res r5 -type test_type1 -attr "PATH_NAME=/tmp/r5.txt"
    crsctl add res r6 -type test_type1 -attr "PATH_NAME=/tmp/r6.txt"

    This adds resources named r5 and r6 to be monitored by Clusterware.
(5) Start/stop the resource using the CRSCTL utility. The commands to do so are:

    crsctl start res r5
    crsctl start res r6
    crsctl check res r5
    crsctl stop res r6
    etc...

    The files /tmp/r5.txt and /tmp/r6.txt get created and deleted as the resources 
    get started and stopped.

Test Macros
-----------

The CRS agent framework provides a set of predefined macros that can be used to 
automatically execute a test suite for a resource. This testing can be done independent 
of the Clusterware installation. The macros are exposed via the header file clsagfwm.h. 
These macros can be used to add a new type, add attributes, start/stop resources etc. 
The file macrodemo.cpp provides an example of this usage. The basic functionality of 
this program is very similar to the demoagent programs. It can be compiled and tested as described 
above. However, it also includes extra functionality to run some unit tests on the added 
resources. To view this functionality, run the program manually with the '-t <testname>' switch 
i.e. "./macrodemo -t dummy". This switch runs the test suite defined in the program to 
automatically add attributes, start/stop resources etc.

Further Information
-------------------

The following resources can be used to get more information on Oracle Clusterware and 
related technology.

(1) The Oracle Clusterware Administration and Deployment Guide - 11g Release 2
(2) The Oracle Technology Network - available online at http://www.oracle.com/technology
